Dynamic interactive objects

ABSTRACT

Aspects of the systems and methods are described providing for modifying a presented interactive element or object, such as a cursor, based on user-input gestures, the presented environment of the cursor, or any combination thereof. The color, size, shape, transparency, and/or responsiveness of the cursor may change based on the gesture velocity, acceleration, or path. In one implementation, the cursor “stretches” to graphically indicate the velocity and/or acceleration of the gesture. The display properties of the cursor may also change if, for example, the area of the screen occupied by the cursor is dark, bright, textured, or is otherwise complicated. In another implementation, the cursor is drawn using sub-pixel smoothing to improve its visual quality.

RELATED APPLICATION

This application is a continuation of U.S. Nonprovisional applicationSer. No. 14/281,825, entitled “Dynamic Interactive Objects”, filed 19May 2014, which claims the benefit of U.S. Provisional Application No.61/824,674, titled “Dynamic Cursors”, filed 17 May 2013. Theseapplications are hereby incorporated by reference for all purposes.

FIELD OF THE TECHNOLOGY DISCLOSED

The technology disclosed generally relates to gesture recognition and,in particular, to providing visual feedback to users based on recognizedgestures.

BACKGROUND

The subject matter discussed in the background section should not beassumed to be prior art merely as a result of its mention in thebackground section. Similarly, a problem mentioned in the backgroundsection or associated with the subject matter of the background sectionshould not be assumed to have been previously recognized in the priorart. The subject matter in the background section merely representsdifferent approaches, which in and of themselves may also correspond toimplementations of the claimed technology.

Traditionally, users have interacted with electronic devices, such as acomputer or a television, or computing applications, such as computergames, multimedia applications, or office applications, via indirectinput devices, including, for example, keyboards, joysticks, or remotecontrollers. Electronics manufacturers have developed systems, however,that detect a user's movements or gestures and cause the display torespond in a contextually relevant manner. The user's gestures may bedetected using an optical imaging system, and characterized andinterpreted by suitable computational resources. For example, a usernear a TV may perform a sliding hand gesture, which is detected by thegesture-recognition system; in response to the detected gesture, the TVmay activate and display a control panel on the screen, allowing theuser to make selections thereon using subsequent gestures; for example,the user may move her hand in an “up” or “down” direction, which, again,is detected and interpreted to facilitate channel selection.

Existing systems, however, provide limited feedback to users in responseto their gestures, and users may find these systems frustrating and hardto use. A system that requires or allows many gestures, or complicatedgestures, may have difficulty in distinguishing one gesture fromanother, and a user may accidentally issue one gesture-based commandwhile intending another. Further, gesture-based presentationenvironments may be complicated, involving many objects, items, andelements, and the user may be confused as to what effect his gestureshave or should have on the system. Hardly the “minority report”interface that we've all been promised.

A need therefore exists for improved technology for providing visualfeedback to the user based on his or her gestures.

SUMMARY

Aspects of the technology described herein provide for modifying apresented (e.g., displayed on a screen, projected in space,holographically projected in space, etc.) interactive element or object,(e.g., a cursor, arrow, representation of a hand or other “real world”object, “window” associated with an operating system or other computerprogram, virtualized representation of one or more documents), based onuser-input gestures, the environment of the presentation area or volume(e.g., cluttered with other objects, darkened, brightened, etc.), or anycombination thereof. A presentation property (e.g., visual, such as thecolor, size, shape, transparency, or audio, such as volume, tone, and/orother, such as responsiveness) of the interactive object may be changedbased on the gesture velocity, acceleration, or path. In animplementation employing a 2D display screen as the presentation medium,a cursor “stretches” to graphically indicate the velocity and/oracceleration of the gesture. The display properties of the cursor may bechanged if, for example, the area of the screen (or volume of a 3Dpresentation space) occupied by the cursor is dark, bright, textured, oris otherwise complicated. In another implementation, the cursor is drawnusing sub-pixel smoothing to improve its visual quality.

In one aspect, a system provides visual feedback in response to agesture. An image-capture device obtains digital images of a real-worldobject in 3D space, and a computer memory stores a digitalrepresentation of a path of movement of the real-world object based onthe captured images. A processor can execute computer code for changinga position of a presented interactive object in response to the movementof the real-world object and modifying a presentation property of thepresented interactive object based on a property of the movement.

A display or other presentation device (e.g., projectors, holographicpresentation systems, etc.) may display the interactive object. Forexample, the interactive object may be a cursor and the real-worldobject may be a hand of a user. Modifying the presentation property ofthe presented interactive object may include changing its color, size,shape, or transparency, stretching the object based on its velocity oracceleration, changing a color of the presented object based on itsvelocity or acceleration, or changing a color of the object based on acolor, texture, or pattern of the screen area along the path. Theproperty of the movement may include a velocity of the real-world objectalong the path, an acceleration of the real-world object along the path,or the screen area along the path. The property of the movement mayinclude a latency between movement of the real-world object and movementof the presented interactive object. Modifying the presentation propertyof the presented interactive object can include stretching the presentedobject an amount related to the latency. Detecting that the path ofmovement does not correspond to a known gesture may cause the processorto perform modifying a presentation property of the presentedinteractive object comprises flashing a color. Detecting a presentedelement to which the presented interactive object is moving towards maycause the processor to perform changing a color of, or flashing, thepresented interactive object the presented element.

In another aspect, a method for providing visual feedback in response toa gesture includes obtaining, using an image-capture device, digitalimages of a real-world object in 3D space; storing, in a computermemory, a digital representation of a path of movement of the real-worldobject based on the captured images; changing a position of a presentedinteractive object in response to the movement of the real-world object;and modifying a presentation property of the presented interactiveobject based on a property of the movement.

Advantageously, these and other aspects enable machines, computersand/or other types of intelligent devices, and/or other types ofautomata to obtain information about objects, events, actions, and/orusers employing gestures, signals, and/or other motions conveyingmeaning and/or combinations thereof. These and other advantages andfeatures of the implementations herein described, will become moreapparent through reference to the following description, theaccompanying drawings, and the claims. Furthermore, it is to beunderstood that the features of the various implementations describedherein are not mutually exclusive and can exist in various combinationsand permutations.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, like reference characters generally refer to like partsthroughout the different views. Also, the drawings are not necessarilyto scale, with an emphasis instead generally being placed uponillustrating the principles of the technology disclosed. In thefollowing description, various implementations of the technologydisclosed are described with reference to the following drawings, inwhich:

FIG. 1 is a simplified block diagram of an exemplary task environment inaccordance with implementations of the technology disclosed;

FIG. 2 is a simplified block diagram of an exemplary system fordetecting a user gesture and modifying a cursor based thereon inaccordance with implementations of the technology disclosed;

FIG. 3 illustrates various cursors and their relationship with usergestures in accordance with implementations of the technology disclosed;and

FIG. 4 illustrates various examples of cursor stretching in accordancewith implementations of the technology disclosed.

DETAILED DESCRIPTION

A motion-capture system suitable to implement the disclosed technologycan include a camera for acquiring images of an object; a computer forprocessing the images to identify and characterize the object; and acomputer display for displaying information related to theidentified/characterized object. A light source may also be included toilluminate the object. FIG. 1 illustrates an exemplary motion-capturesystem 100. The system 100 includes one or more light-capturing devices102 (e.g., digital cameras or similar devices), each including an imagesensor (e.g., a CCD or CMOS sensor), an associated imaging optic (e.g.,a lens), and a window of transparent material protecting the lens fromthe environment. Two or more cameras 102 may be arranged such that theirfields of view overlap in a viewed region. One or more light-emittingdevices 104 may be used to illuminate an object 106 in the field ofview. The cameras 102 provide digital image data to a computer 108,which analyzes the image data to determine the 3D position, orientation,and/or motion of the object 106 the field of view of the cameras 102.

The cameras 102 may be visible-light cameras, infrared (IR) cameras,ultraviolet cameras, or cameras operating in any other electromagneticfrequency regime. Preferably, the cameras 102 are capable of capturingvideo images. The particular capabilities of cameras 102 may vary as toframe rate, image resolution (e.g., pixels per image), color orintensity resolution (e.g., number of bits of intensity data per pixel),focal length of lenses, depth of field, etc. In general, for aparticular application, any cameras capable of focusing on objectswithin a spatial volume of interest can be used. For instance, tocapture motion of the hand of an otherwise stationary person, the volumeof interest might be a cube of one meter in length. To capture motion ofa running person, the volume of interest might have dimensions of tensof meters in order to observe several strides.

The cameras may be oriented in any convenient manner. In one embodiment,the optical axes of the cameras 102 are parallel, and in otherimplementations the optical axes of the cameras 102 are not parallel. Asdescribed below, each camera 102 may be used to define a “vantage point”from which the object 106 is seen; if the location and view directionassociated with each vantage point are known, the locus of points inspace that project onto a particular position in the camera's imageplane may be determined. In some embodiments, motion capture is reliableonly for objects in an area where the fields of view of cameras 102; thecameras 102 may be arranged to provide overlapping fields of viewthroughout the area where motion of interest is expected to occur. Inother embodiments, the system 100 may include one or more light sources104, and the cameras 102 measure the reflection of the light emitted bythe light sources on objects 106. The system may include, for example,two cameras 102 and one light source 104; one camera 102 and two lightsources 104; or any other appropriate combination of light sources 104and cameras 102.

Computer 108 may generally be any device or combination of devicescapable of processing image data using techniques described herein. FIG.2 is a simplified block diagram of a suitably programmed general-purposecomputer 200 implementing the computer 108 according to implementationsof the technology disclosed. The computer 200 includes a processor 202with one or more central processing units (CPUs), volatile and/ornon-volatile main memory 204 (e.g., RAM, ROM, or flash memory), one ormore mass storage devices 206 (e.g., hard disks, or removable media suchas CDs, DVDs, USB flash drives, etc. and associated media drivers), adisplay device 208 (e.g., a liquid crystal display (LCD) monitor), userinput devices such as keyboard 210 and mouse 212, and one or more buses214 (e.g., a single system bus shared between all components, orseparate memory and peripheral buses) that facilitate communicationbetween these components.

The cameras 102 and/or light sources 104 may connect to the computer 200via a universal serial bus (USB), FireWire, or other cable, orwirelessly via Bluetooth, Wi-Fi, etc. The computer 200 may include acamera interface 216, implemented in hardware (e.g., as part of a USBport) and/or software (e.g., executed by processor 202), that enablescommunication with the cameras 102 and/or light sources 104. The camerainterface 216 may include one or more data ports and associated imagebuffers for receiving the image frames from the cameras 102; hardwareand/or software signal processors to modify the image data (e.g., toreduce noise or reformat data) prior to providing it as input to amotion-capture or other image-processing program; and/or control signalports for transmit signals to the cameras 102, e.g., to activate ordeactivate the cameras, to control camera settings (frame rate, imagequality, sensitivity, etc.), or the like.

The main memory 204 may be used to store instructions to be executed bythe processor 202, conceptually illustrated as a group of modules. Thesemodules generally include an operating system (e.g., a MicrosoftWINDOWS, Linux, or APPLE OS X operating system) that directs theexecution of low-level, basic system functions (such as memoryallocation, file management, and the operation of mass storage devices),as well as higher-level software applications such as, e.g., amotion-capture (mocap) program 218 for analyzing the camera images totrack the position of an object of interest and/or a motion-responseprogram for computing a series of output images (or another kind ofresponse) based on the tracked motion. Suitable algorithms formotion-capture program are described further below as well as, in moredetail, in U.S. patent application Ser. No. 13/414,485, filed on Mar. 7,2012 and Ser. No. 13/742,953, filed on Jan. 16, 2013, and U.S.Provisional Patent Application No. 61/724,091, filed on Nov. 8, 2012,which are hereby incorporated herein by reference in their entirety. Thevarious modules may be programmed in any suitable programming language,including, without limitation high-level languages such as C, C++, C #,OpenGL, Ada, Basic, Cobra, Fortran, Java, Lisp, Perl, Python, Ruby, orObject Pascal, or low-level assembly languages.

The memory 204 may further store input and/or output data associatedwith execution of the instructions (including, e.g., input and outputimage data 220) as well as additional information used by the varioussoftware applications; for example, in some embodiments, the memory 204stores an object library 222 of canonical models of various objects ofinterest. As described below, an object detected in the camera imagesmay identified by matching its shape to a model in the object library222, and the model may then inform further image analysis, motionprediction, etc.

In various embodiments, the motion captured in a series of camera imagesis used to compute a corresponding series of output images for displayon the computer screen 208. For example, camera images of a moving handmay be translated into a wire-frame or other graphic depiction of thehand by the processor 202. Alternatively, hand gestures may beinterpreted as input used to control a separate visual output; by way ofillustration, a user may be able to use upward or downward swipinggestures to “scroll” a webpage or other document currently displayed, oropen and close her hand to zoom in and out of the page. In any case, theoutput images are generally stored in the form of pixel data in a framebuffer, which may, but need not be, implemented in main memory 204. Avideo display controller reads out the frame buffer to generate a datastream and associated control signals to output the images to thedisplay 208. The video display controller may be provided along with theprocessor 202 and memory 204 on-board the motherboard of the computer200, and may be integrated with the processor 202 or implemented as aco-processor that manipulates a separate video memory. In someembodiments, the computer 200 is equipped with a separate graphics orvideo card that aids with generating the feed of output images for thedisplay 208. The video card generally includes a graphical processingunit (“GPU”) and video memory, and is useful, in particular, for complexand computationally expensive image processing and rendering. Thegraphics card may implement the frame buffer and the functionality ofthe video display controller (and the on-board video display controllermay be disabled). In general, the image-processing and motion-capturefunctionality of the system may be distributed between the GPU and themain processor 202 in various conventional ways that are wellcharacterized in the art.

The computer 200 is an illustrative example; variations andmodifications are possible. Computers may be implemented in a variety ofform factors, including server systems, desktop systems, laptop systems,tablets, smart phones or personal digital assistants, and so on. Aparticular implementation may include other functionality not describedherein, e.g., wired and/or wireless network interfaces, media playingand/or recording capability, etc. In some embodiments, one or morecameras may be built into the computer rather than being supplied asseparate components. Further, the computer processor may be ageneral-purpose microprocessor, but depending on implementation canalternatively be, e.g., a microcontroller, peripheral integrated circuitelement, a customer-specific integrated circuit (“CSIC”), anapplication-specific integrated circuit (“ASIC”), a logic circuit, adigital signal processor (“DSP”), a programmable logic device such as afield-programmable gate array (“FPGA”), a programmable logic device(“PLD”), a programmable logic array (“PLA”), smart chip, or other deviceor arrangement of devices.

Further, while computer 200 is described herein with reference toparticular blocks, this is not intended to limit the technologydisclosed to a particular physical arrangement of distinct componentparts. For example, in some embodiments, the cameras 102 are connectedto or integrated with a special-purpose processing unit that, in turn,communicates with a general-purpose computer, e.g., via direct memoryaccess (“DMA”). The processing unit may include one or more imagebuffers for storing the image data read out from the camera sensors, aGPU or other processor and associated memory implementing at least partof the motion-capture algorithm, and a DMA controller. The processingunit may provide processed images or other data derived from the cameraimages to the computer for further processing. In some embodiments, theprocessing unit sends display control signals generated based on thecaptured motion (e.g., of a user's hand) to the computer, and thecomputer uses these control signals to adjust the display of documentsand images that are otherwise unrelated to the camera images (e.g., textdocuments or maps) by, for example, shifting or rotating the images.

In one embodiment, the system 200 displays an interactive object, suchas a cursor, on the screen 208; a user may change the position,orientation, or other display property of the cursor on the screen 208(in 2D or 3D space) by making a gesture with a real-world object, suchas the user's hand, finger, other body part, an object held in theuser's hand, or any other object. A gesture-recognition module 224tracks the gesture and translates the gesture into a movement command;an interactive object presentation module 226 moves, rotates, orotherwise alters the appearance or responsiveness of the presentation ofthe interactive object accordingly. For example, if the user moves hisor her hand to the left, the system moves the cursor on the screen tothe left by virtue of operation of modules 224 and 226.

The interactive object presentation module 226 may alter the appearanceand/or behavior of the cursor based on a number of factors, includingthe velocity, acceleration, or path of the tracked gesture; the screenenvironment of the cursor; or the past or predicted future movement ofthe tracked object. As the term is used herein, “velocity” may mean avector representing the speed and direction of motion of the gesture orsimply the speed of the gesture. The cursor may be an arrow, circle,ellipse, or any other type of pointer or graphic; the cursor may also bea realistic or stylized representation of a user's hand, stylus, orother object. The technology disclosed is not limited to only cursors,however, and the implementations described herein may be used with otherinteractive elements, such as scrollbars, sliders, knobs, buttons, orany other type of interactive element. FIG. 3 illustrates a system 300that includes a display 302 and a user's hand 304. When the hand 304moves in a direction 306, exemplary cursors 308, 310, 312correspondingly move. The cursor may be an “arrow”-style cursor 308, anellipse 310, or any other shape; these cursors move on the screen 302 ina direction 314 corresponding to the direction of motion 306 of the hand304. The amount presented motion 314 of the cursors 308, 314 may beequal to, less than, or greater than the amount of real-world motion 306of the hand 304. In other implementations, the cursor may be anotherpresented element, such as a dial 312 that exhibits other types ofmovement, such as a rotation 316, in response to movement of the hand304.

In one implementation, the interactive object presentation module 226changes the appearance of a cursor based on the velocity of the trackedobject. For example, a first color may be assigned to the cursor, or apart of the cursor, when the tracked object is not moving, and a secondcolor may be assigned to the cursor, or a part of the cursor, when thetracked object is moving above a threshold rate of speed. The first andsecond colors may be blended to varying degrees to representintermediate rates of speeds as the object accelerates from not movingto above a threshold rate of speed. For example, the first color may bered which may gradually transition to green as the speed of the trackedobject increases. In another embodiment, the color changes more quicklyor instantly when the speed crosses a threshold. For example, the colormay be red while the object is at rest or moving slowly to indicate tothe user that no command is currently associated with the gesture; whenthe speed of the object passes the threshold, however, the color turnsto green to indicate to the user that the gesture has been recognizedand an associated command is thereupon executed by the system 200. Moregenerally, the color of the cursor may change in response to any motionparameter associated with the presented object. Motion parametersincludes velocity, acceleration, and changes in acceleration. A firstcolor may denote an increase in a particular parameter while a secondcolor indicates a decrease. The motion parameter illustrated by way ofcolor changes may depend on instantaneous analysis of the gesture; forexample, if the rate of acceleration crosses a threshold, the“acceleration” color scheme may replace the “velocity” color scheme.Furthermore, the term “color” is herein used broadly to connoteintensity, hue, value, chroma or other property that may increase ordecrease in accordance with the associated motion parameter.

In various implementations, multiple cursors are used to track multipleobjects. For example a plurality of fingers on a user's hands may betracked and each may have an associated cursor. The interactive objectpresentation module 226 applies different colors to some or all of themultiple cursors. For example, if a current application running on thesystem 200 uses only the index fingers of the user's hands as inputs,the interactive object presentation module 226 may display presentedhand representations corresponding to the user's hands in which theindex fingers of the presented hands are colored a first color (e.g.,green) while the rest of the fingers are a second color (e.g., red). Theinteractive object presentation module 226 may change these colorassignments if and when the application accepts input from otherfingers. If the user is holding a stylus or other input device in his orher hand, the stylus may be colored a first color while the hand iscolored a second color.

The interactive object presentation module 226 may change the color ofthe cursor when the gesture-recognition module 224 cannot recognize auser gesture. In implementations the cursor flashes a different color,for example red, if the user makes a gesture that thegesture-recognition module 224 cannot recognize. In addition, inimplementations the interactive object presentation module 226 maychange the color of the cursor based on the current use mode of thecursor. In an implementation, if the cursor is in a first mode, forexample a “movement” mode where the user intends to move the cursor butnot interact with any presented elements, the cursor may be a firstcolor, and when the user is in a second mode, for example an “action”mode where the user intends to move, click on, or otherwise interactwith a presented element, the cursor may be a second color. Theinteractive object presentation module 226 may select a current mode ortransition to a different mode based on analysis of the user's patternof action, in response to a command from a running application, or uponuser command. The interactive object presentation module 226 may changethe shape of the cursor in accordance with the motion, includingvelocity and/or acceleration, of the tracked object. In oneimplementation, the cursor “stretches” by expanding in size in one ormore dimensions) in response to motion of the object. For example, thecursor may expand in a direction approximately opposite the direction ofmotion of the cursor, so that the user's motion seems to be stretchingthe cursor. The stretching may increase the size of the cursor (i.e.,the screen area occupied by the user). In another implementation, as thecursor stretches in one direction, it shrinks in another dimension suchthat the total screen area occupied by the cursor remains approximatelyconstant before, during, and after the stretching. The amount ofstretching may be proportional to the velocity and/or acceleration; inone embodiment, stretching of the cursor occurs only after the velocityor acceleration of the object increases past a threshold. The amountstretching may be controlled only by the acceleration of the object, inwhich case constant-velocity motion of the object produces nostretching. When the object decelerates, the amount of stretchingdecreases until the cursor returns to the size it was at rest. In someembodiments, deceleration causes the cursor to shrink smaller than itssize at rest. FIG. 4 illustrates a system 400 that includes a screen 402and a user's hand 404; an ellipse-based cursor 406A stretches to a firstnew shape 406B in response to a first increase in velocity oracceleration 408 of the user's hand 404, and moves a correspondingamount 410. The first new shape 406B stretches to a second new shape406C in response to further increases in velocity or acceleration 412 ofthe user's hand 404 and moves a greater amount 414. In this embodiment,the vertical dimension of the ellipses 406A, 406B, 406C decreases as thehorizontal dimension increases, thereby maintaining a roughly constantarea of the ellipses 406A, 406B, 406C; in other embodiments, thevertical dimension of the ellipses 406A, 406B, 406C changes in otherdirections or amounts or does not change at all.

Latency may exist between the time a tracked object moves and the timeat which the cursor reacts on the screen 208. That is, the system 200may require a certain amount of time to detect the movement of thetracked object with the cameras 102, process and recognize the gesturerelated to the movement with the processor 202, and send an update tothe screen 208. This amount of time, referred to as the latency of thesystem 200, may be approximately 10-100 ms and may vary in response tothe current level of processing power available on the processor 202,the refresh rate of the screen 208, or other factors. In one embodiment,the interactive object presentation module 226 stretches the cursor inaccordance with the latency between a tracked movement of the object andthe corresponding motion of the cursor. The stretching, by giving theappearance of faster motion, may lessen the amount of latency that theuser perceives. In one embodiment, the amount of stretching isproportional to the current amount of latency. The system 200 maydetermine the latency by measuring some or all of it by, for example,measuring the current processing speed of the processor 202 or may use apredetermined or expected latency.

In one implementation, the system 200 changes a display property of thecursor based on the context of the cursor on the screen 208. If, forexample, the area surrounding the cursor on the screen 208 is dark, theinteractive object presentation module 226 may increase the brightnessof the cursor to help the user identify or locate it or may decrease thebrightness of the cursor so that it fits more naturally into itssurroundings. Similarly, if the screen is bright, the interactive objectpresentation module 226 may increase or decrease the brightness of thecursor. The interactive object presentation module 226 may instead or inaddition adjust the contrast or transparency (i.e., alpha level) of thecursor in response to the brightness of its surroundings.

The color, pattern, texture, and/or motion of the area surrounding thecursor on the screen 208 may also be used to change the displayproperties of the cursor. In one embodiment, the interactive objectpresentation module 226 analyzes any objects, applications, orbackgrounds displayed on the screen 208 near the cursor and possibly thecurrent display properties of the cursor itself to determine the degreeof similarity there between. For example, the cursor may move to aregion of the screen 208 having the same color as the cursor, and theuser may experience difficulty in distinguishing the cursor from thesimilarly colored region. Likewise, the region of the screen 208 mayhave a similar pattern or texture as the cursor or, even if the cursor'spattern or texture is dissimilar, the pattern or texture of the regionof the screen 208 may be complicated or confusing to the user. In oneembodiment, the interactive object presentation module 226 alters thecolor, transparency, contrast, pulsing, flashing, or other property ofthe cursor to thereby aid the user in distinguishing the cursor from theregion of the screen.

The interactive object presentation module 226 may change theresponsiveness of the cursor based on the gesture and/or screen regionnear the cursor. In one embodiment, as the cursor nears an interactiveelement on the screen 208, the interactive object presentation module226 decreases the sensitivity of the cursor with respect to the movementof the object. For example, if the cursor approaches a presented button,scrollbar, URL link, or any other such object, a gesture of a givenlength or rotation causes the cursor to move a shorter distance than itwould otherwise (if, e.g., the cursor is not near such an interactiveelement). In one implementation, the gesture-recognition module 224determines that the user intends to interact with the presented elementbased on the speed or direction of a detected gesture; in thisimplementation, the interactive object presentation module 226 “snaps”the cursor to the object and the cursor quickly or instantly moves tothe presented element. The interactive object presentation module 226may further treat the cursor as “sticky” once it moves to or interactswith the presented element and the cursor remains on or near the elementuntil the user makes a gesture having a velocity, distance, oracceleration greater than a threshold.

In another implementation, the interactive object presentation module226 causes the cursor to be displayed on the screen 208 using sub-pixelrendering or smoothing. The interactive object presentation module 226may detect or receive physical properties of the display 208 (e.g.,screen resolution and LED/OLED type) and use these properties to rendersub-pixels on the display. The sub-pixel rendering may provideanti-aliasing to the cursor, thereby increasing the perceived resolutionof the cursor.

The interactive object presentation module 226 may change thepresentation property of the cursor based on past or predicted futuremovement of the real-world object. For example, if the cursor is used torotate or move an element such as dial, scrollbar, or slider (or thecursor is a dial, scrollbar, or slider), the cursor or element may flashand/or change color when the element approaches its 25%, 50%, 75%, or100% range of motion. If the cursor is moving toward an interactiveelement, the interactive object presentation module 226 may change theappearance of the cursor to indicate interaction with the element. Forexample, if the cursor is moving toward a URL link, it may turn blue tomatch the color of the link. In another embodiment, the interactiveelement flashes or changes color when the cursor moves toward it. Theinteractive object presentation module 226 may indicate past movement ofthe cursor by showing, e.g., a line that tracks the past movement.

Implementations may employed in a variety of application areas, such asfor example and without limitation consumer applications includinginterfaces for computer systems, laptops, tablets, television, gameconsoles, set top boxes, telephone devices and/or interfaces to otherdevices; medical applications including controlling devices forperforming robotic surgery, medical imaging systems and applicationssuch as CT, ultrasound, x-ray, MRI or the like, laboratory test anddiagnostics systems and/or nuclear medicine devices and systems;prosthetics applications including interfaces to devices providingassistance to persons under handicap, disability, recovering fromsurgery, and/or other infirmity; defense applications includinginterfaces to aircraft operational controls, navigations systemscontrol, on-board entertainment systems control and/or environmentalsystems control; automotive applications including interfaces toautomobile operational systems control, navigation systems control,on-board entertainment systems control and/or environmental systemscontrol; security applications including, monitoring secure areas forsuspicious activity or unauthorized personnel; manufacturing and/orprocess applications including interfaces to assembly robots, automatedtest apparatus, work conveyance devices such as conveyors, and/or otherfactory floor systems and devices, genetic sequencing machines,semiconductor fabrication related machinery, chemical process machineryand/or the like; and/or combinations thereof.

Implementations of the technology disclosed may further be mounted onautomobiles or other mobile platforms to provide information to systemstherein as to the outside environment (e.g., the positions of otherautomobiles). Further implementations of the technology disclosed may beused to track the motion of objects in a field of view or used inconjunction with other mobile-tracking systems. Object tracking may beemployed, for example, to recognize gestures or to allow the user tointeract with a computationally rendered environment; see, e.g., U.S.Patent Application Ser. No. 61/752,725 (filed on Jan. 15, 2013) and Ser.No. 13/742,953 (filed on Jan. 16, 2013), the entire disclosures of whichare hereby incorporated by reference.

It should also be noted that implementations of the technology disclosedmay be provided as one or more computer-readable programs embodied on orin one or more articles of manufacture. The article of manufacture maybe any suitable hardware apparatus, such as, for example, a floppy disk,a hard disk, a CD ROM, a CD-RW, a CD-R, a DVD ROM, a DVD-RW, a DVD-R, aflash memory card, a PROM, a RAM, a ROM, or a magnetic tape. In general,the computer-readable programs may be implemented in any programminglanguage. Some examples of languages that may be used include C, C++, orJAVA. The software programs may be further translated into machinelanguage or virtual machine instructions and stored in a program file inthat form. The program file may then be stored on or in one or more ofthe articles of manufacture.

Certain implementations of the technology disclosed were describedabove. It is, however, expressly noted that the technology is notlimited to those implementations, but rather the intention is thatadditions and modifications to what was expressly described herein arealso included within the scope of the technology disclosed. For example,it may be appreciated that the techniques, devices and systems describedherein with reference to examples employing light waves are equallyapplicable to methods and systems employing other types of radiantenergy waves, such as acoustical energy or the like. Moreover, it is tobe understood that the features of the various embodiments describedherein were not mutually exclusive and can exist in various combinationsand permutations, even if such combinations or permutations were notmade express herein, without departing from the spirit and scope of thetechnology disclosed. In fact, variations, modifications, and otherimplementations of what was described herein will occur to those ofordinary skill in the art without departing from the spirit and thescope of the technology disclosed.

What is claimed is:
 1. A system comprising: a computer memory configuredto store a digital representation of a path of a movement of two or morefingers of a user's hands tracked in a 3D space; and a processorimplementing: presenting, on a presentation device, separate interactiveobjects, each separate interactive object being associated with aparticular finger of two or more fingers, wherein a position of eachpresented interactive object changes in response to a movement of theassociated finger of the two or more fingers; and determining a use modefor at least one of the presented interactive objects and modifying afirst presentation property of the at least one of the presentedinteractive objects based on the use mode determined for the at leastone of the presented interactive objects, wherein the use mode isdetermined based on an input from an application or an input from auser, wherein the use mode is determined to be one of: a first mode inwhich the at least one of the presented interactive objects moves butdoes not interact with any presented elements; and a second mode inwhich the at least one of the presented interactive objects interactswith a presented element, and wherein the at least one of the presentedinteractive objects is at least one of a cursor, a scrollbar, a slider,a knob and a button.
 2. The system of claim 1, further comprising adisplay configured to display the presented interactive objects.
 3. Thesystem of claim 1, wherein the processor further implements modifying asecond presentation property selected from a set consisting of a color,a size and a shape of at least one of the presented interactive objectsbased on a determined property of the movement.
 4. The system of claim3, wherein the modifying of the second presentation property of the atleast one of the presented interactive objects comprises changing atleast one of a color, a size, a shape, and a transparency of the atleast one of the presented interactive objects.
 5. The system of claim3, wherein the modifying of the second presentation property of the atleast one of the presented interactive objects comprises at least one ofstretching the at least one of the presented interactive objects basedon at least one of a velocity and an acceleration of at least one of thetwo or more fingers of the user's hands, changing a color of the atleast one of the presented interactive objects based on at least one ofthe velocity and the acceleration of the at least one of the two or morefingers of the user's hands, and changing a color of the at least one ofthe presented interactive objects based on at least one of a color, atexture, and a pattern of a screen area along the path.
 6. The system ofclaim 3, wherein the processor further implements detecting that thepath of the movement does not correspond to a known gesture, and whereinthe modifying of the second presentation property of the at least one ofthe presented interactive objects comprises changing a color of the atleast one of the presented interactive objects in response to detectingthat the path of the movement does not correspond to the known gesture.7. The system of claim 1, wherein the processor further implementsdetecting a presented element to which at least one of the presentedinteractive objects is moving towards and changing a color of at leastone of the presented interactive objects to indicate interaction withthe presented element.
 8. A method comprising: storing, in a computermemory, a digital representation of a path of a movement of two or morefingers of a user's hands tracked in a 3D space; presenting, on apresentation device, separate interactive objects, each separateinteractive object being associated with a particular finger of two ormore fingers, wherein a position of each presented interactive objectchanges in response to a movement of the associated finger of the two ormore fingers; and determining a use mode for at least one of thepresented interactive objects and modifying a first presentationproperty of the at least one of the presented interactive objects basedon the use mode determined for the at least one of the presentedinteractive objects, wherein the use mode is determined based on aninput from an application or an input from a user, wherein the use modeis determined to be one of: a first mode in which the at least one ofthe presented interactive objects moves but does not interact with anypresented elements; and a second mode in which the at least one of thepresented interactive objects interacts with a presented element, andwherein the at least one of the presented interactive objects is atleast one of a cursor, a scrollbar, a slider, a knob and a button. 9.The method of claim 8, further comprising displaying, on a display, thepresented interactive objects.
 10. The method of claim 8, furthercomprising modifying a second presentation property selected from a setconsisting of a color, a size, and a shape of at least one of thepresented interactive objects based on a determined property of themovement.
 11. The method of claim 10, wherein the modifying of thesecond presentation property of the at least one of the presentedinteractive objects comprises changing at least one of a color, a size,a shape, and a transparency of the at least one of the presentedinteractive objects.
 12. The method of claim 10, wherein the modifyingof the second presentation property of the at least one of the presentedinteractive objects comprises at least one of: stretching the at leastone of the presented interactive objects based on at least one of avelocity and an acceleration of at least one of the two or more fingersof the user's hands, changing a color of the at least one of thepresented interactive objects based on at least one of the velocity andthe acceleration of the at least one of the two or more fingers of theuser's hands, and changing a color of the at least one of the presentedinteractive objects based on at least one of a color, a texture, and apattern of a screen area along the path.
 13. The method of claim 10,further including detecting that the path of the movement does notcorrespond to a known gesture, wherein the modifying of the secondpresentation property of the at least one of the presented interactiveobjects comprises changing a color of the at least one of the presentedinteractive objects in response to detecting that the path of themovement does not correspond to the known gesture.
 14. The method ofclaim 8, further including detecting a presented element to which atleast one of the presented interactive objects is moving towards andwherein the modifying of the first presentation property of the at leastone of the presented interactive objects to indicate interaction withthe presented element.
 15. The method of claim 8, wherein at least oneof the presented interactive objects is presented in a first color whenthe at least one of the presented interactive objects is in a first usemode; and the at least one of the presented interactive objects ispresented in a second color when the at least one of the presentedinteractive objects is in a second use mode, wherein the first andsecond colors are visually distinct.
 16. The method of claim 8, furtherincluding selecting a current use mode based upon a user's pattern ofaction.
 17. The method of claim 8, further including selecting a currentuse mode based upon a command received from a running application. 18.The method of claim 8, further including selecting a current use modebased upon a command received from a user.
 19. The method of claim 8,further including clicking on the presented element when the at leastone of the presented interactive objects interacts with the presentedelement.
 20. A method comprising: storing, in a computer memory, adigital representation of a path of a movement of two or more fingers ofa user's hands tracked in a 3D space for presentation on a screen;determining a current use mode for a presented interactive objectdisplayed on the screen and associated with one of the two or morefingers as being an action mode, in which the presented interactiveobject interacts with a presented element displayed on the screen butdoes not move from the presented element, wherein the determination ofthe current use mode is in response to a command from a runningapplication; and modifying a first presentation property of thepresented interactive object based on the determined current use mode ofthe presented interactive object; wherein, when the determined currentuse mode is the action mode, the presented interactive object interactswith a presented element displayed on the screen in response to a usercommand by at least one of the two or more fingers of the user's hands,wherein the presented interactive object is presented in a first colorwhen the presented interactive object is in a first use mode, thepresented interactive object is presented in a second color when thepresented interactive object is in a second use mode, and wherein thefirst and second colors are visually distinct, and wherein the presentedinteractive object is at least one of a cursor, a scrollbar, a slider aknob and a button.